Adversarial examples in random neural networks with general activations

A substantial body of empirical work documents the lack robustness in deep learning models to adversarial examples. Recent theoretical proved that examples are ubiquitous two-layers networks with sub-exponential width and ReLU or smooth activations, multi-layer width. We present a result same type, no restriction on for general locally Lipschitz continuous activations. More precisely, given neural network $f(,\cdot,;\mathbf{\theta})$ random weights $\mathbf{\theta}$, feature vector $\mathbf{x}$, we show an example $\mathbf{x}'$ can be found high probability along direction gradient $\nabla\_{\mathbf{x}}f(\mathbf{x};\mathbf{\theta})$. Our proof is based Gaussian conditioning technique. Instead proving $f$ approximately linear neighborhood characterize joint distribution $f(\mathbf{x};\mathbf{\theta})$ $f(\mathbf{x}';\mathbf{\theta})$ $\mathbf{x}' = \mathbf{x}-s(\mathbf{x})\nabla\_{\mathbf{x}}f(\mathbf{x};\mathbf{\theta})$, where $s(\mathbf{x}) \operatorname{sign}(f(\mathbf{x}; \mathbf{\theta})) \cdot s\_d$ some positive step size $s\_d$.